Methodology of Evaluation in Medical Image Computing

About one decade ago, Yannis Aloimonos complained that \Unfortunately, there is a disconcerting lack of visual systems which perform well in real-world environments, particularly when compared to the amount of mathematical theory published on the subject."| a complaint which not only holds for computational vision in general but in particular also for the safety-critical case of medical image computing (a terminus technicus which commonly subsumes medical image formation, processing, analysis, interpretation, and visualization). One reason for this unfortunate situation is clearly the fact that the experimental basis of computational vision as a scienti c discipline is still rather weak. As a down-to-earth-consequence, e.g., it is by no means clear for an industrial system designer, on which grounds she/he should rely on a particular algorithm, method, or proposed tool once she/he is faced with the problem of putting academic research to work. Neither it seems to be clear for a clinician, what kind of as well as what degree or quality of support in his routine work she/he can expect from pro ered medical image computing (MIC) tools claimed to support routine work. Put in other words, MIC seen as a coin has a shiny and scienti cally rewarding theory side but a rather rusty, not to say puny, practice side. Meanwhile in the MIC community a growing awareness of the fact can be observed that evaluation aiming at performance characterization is a critical issue. In a complementing way, a strong need from both clinical and industrial actors for tackling theoretical as well as experimental problems associated with this issue has to be stated, since dissemination of theoretical advances into practical settings requires a deep understanding of assets, limitations, application scope, etc. of MIC algorithms, methods, and tools. Moreover it is safe to state that without such a deep understanding gained from a scienti c approach the design of interactive MIC systems will be severely hampered, since human-centered eÆcient interaction should take place on the basis of results of computational processes which are trustworthy|ideally results, which are consistent with theoretical proofs of a computational theory. In contrast to other application domains drawing upon visual data, MIC stands out for reasons of required safety, accuracy, robustness, ergonomy, etc. Apart from that, MIC is seen as a major future high-tech market also, hence the development of successful products strongly depends on bridging the gap between theory, experiment, and practice. Obviously, solutions to these problems reside in a space composed of multiple dimensions to name a few: MIC theory, practice of MIC (incl. design of algorithms and visual data structures), clinical requirement analysis, and industrial platform constraints. Due to the lack of well-grounded, internationally accepted, and standardized methods for evaluation and given the speci ty of MIC as brie y sketched above, it was high-time to bring together leading experts from the MIC community in the inspiring atmosphere of Schloss Dagstuhl to discuss the state-of-the-art/technology as well as routes to be jointly